Skip to content

Record reports after the upload, not before - #347

Merged
sjoelund merged 1 commit into
masterfrom
record-reports-after-upload
Aug 31, 2026
Merged

Record reports after the upload, not before#347
sjoelund merged 1 commit into
masterfrom
record-reports-after-upload

Conversation

@sjoelund

Copy link
Copy Markdown
Member

all-reports.py wrote a report's row into [history] as soon as it had written the html to the workspace, and mailed everyone about it in the same run. The files only reach the web server at the end of the stage, after all-plots.py and two dozen report.py invocations, so anything failing in between left a report that the database says exists, that nobody can read, and that the rerun skips because it is recorded. The mail linking to it had already gone out.

Before the table existed the record was the published 00_history.html, which only existed if the upload had happened, so a lost upload healed itself on the next run; moving the record into the database made it permanent.

The reports and the queued emails now go into pending-reports.json, and publish-reports.py records and sends them once the files are up. The stage uploads history/ right after all-plots.py and keeps a second transfer for overview*.html at the end.

scenario now
generate, publish, rerun skipped
stage fails before the upload regenerated next run
uploaded but not recorded copied back from the index
branch named twice in one run one report

Assisted-by: Claude Opus 5

all-reports.py wrote a report's row into [history] as soon as it had
written the html to the workspace, and mailed everyone about it in the
same run. The files only reach the web server at the end of the stage,
after all-plots.py and two dozen report.py invocations, so anything
failing in between left a report that the database says exists, that
nobody can read, and that the rerun skips because it is recorded. The
mail linking to it had already gone out.

Before the table existed the record was the published 00_history.html,
which only existed if the upload had happened, so a lost upload healed
itself on the next run; moving the record into the database made it
permanent.

The reports and the queued emails now go into pending-reports.json, and
publish-reports.py records and sends them once the files are up. The
stage uploads history/ right after all-plots.py and keeps a second
transfer for overview*.html at the end.

| scenario                            | now                          |
| ----------------------------------- | ---------------------------- |
| generate, publish, rerun            | skipped                      |
| stage fails before the upload       | regenerated next run         |
| uploaded but not recorded           | copied back from the index   |
| branch named twice in one run       | one report                   |

Assisted-by: Claude Opus 5
@sjoelund
sjoelund enabled auto-merge (squash) August 31, 2026 18:36
@sjoelund
sjoelund merged commit aa27e41 into master Aug 31, 2026
7 checks passed
@sjoelund
sjoelund deleted the record-reports-after-upload branch August 31, 2026 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant